This Page Is Inserted by IFW Operations 
and is not a part of the Official Record 


BEST AVAILABLE IMAGES 

Defective images within this document are accurate representations of 
the original documents submitted by the applicant. 

Defects in the images may include (but are not limited to): 

• BLACK BORDERS 

• TEXT CUT OFF AT TOP, BOTTOM OR SIDES 

• FADED TEXT 

• ILLEGIBLE TEXT 

• SKEWED/SLANTED IMAGES 

• COLORED PHOTOS 

• BLACK OR VERY BLACK AND WHITE DARK PHOTOS 

• GRAY SCALE DOCUMENTS 


IMAGES ARE BEST AVAILABLE COPY. 


As rescanning documents will not correct images, 
please do not report the images to the 
Image Problem Mailbox. 


THIS PAGE BLANK (USPTO) 



DOSSIER 

(12) INTERNATIONAL /^PLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 

(19) ^>^£^S^ Dtetta llllllllllllilllllilllllll 

(43) International Publication Date (10) International Publication Number 

7 June 2001 (07.06.2001) PCT WO 01/40938 A2 

(51) International Patent Classification 7 : G06F 9/46 Prof. Holstlaan 6, NL-5656 AA Eindhoven (NL). 

KRAS1NSKI, Raymond; Prof. Holstlaan 6. NL-5656 


(21) International Application Number: PCT/EPO0/1 1412 


AA Eindhoven (NL). 


_ 4 (74) Agent: GROENENDAAL, Antonius, W., M.; Interna- 

(22) International Filing Da e: _ , , _ , tionaa! Octrooibureau B. V.. Prof Holstlaan 6, NL-5656 AA 


1 6 November 2000 ( 1 6. 11 .2000) 
(25) Filing Language: English 


Eindhoven (NL). 
(81) Designated States (national)'. CN, JP, KR. 


(26) Publication Language: English (g4) Designated States (regional): European patent (AT, BE, 

CH, CY, DE, DK, ES, Fl, FR, GB, GR, IE, IT, LU, MC, 
(30) Priority Data: NL. PT, SE). 

09/454,348 3 December 1999 (03.12.1999) US 

Published: 

(71) Applicant: KONINKLIJKE PHILIPS ELECTRON- — Without international search report and to be republished 
ICS N.V. [NL/NL1; Groenewoudseweg 1, NL-5621 BA upon receipt of that report. 

Eindhoven (NL). 

For two-letter codes and other abbreviations, refer to the "Guid- 
:== (72) Inventors: ALSAFADI, Yasser; Prof. Holstlaan 6, ance Notes on Codes and Abbreviations" appearing at the begin- 
= NL-5656 AA Eindhoven (NL). DASGUPTA, Aninda; ning of each regular issue of the PCT Gazette. 


= (54) Title: METHODS FOR INITIATING ACTIVITY IN INTELLIGENT DEVICES CONNECTED TO AN IN HOME DIGI- 
Z== TAL NETWORK USING EXTENSIBLE MARKUP LANGUAGE (XML) FOR INFORMATION EXCHANGE AND SYSTEMS 
THEREFOR 


fs| (57) Abstract: A method of operating a system including a digital network interconnected intelligent digital devices (IDDs) gen- 

^ erating and receiving extensible Markup Language (XML) documents containing data and respective Document Type Definitions 
(DTDs) describing the data content. Preferably, the method includes steps for transmitting a generated XML document from a first 

00 IDD to a second IDD, and, when the respective DTD for the generated XML document satisfies a predetermined criteria, operating 

f*} on the data contained in the XML document at the second IDD. In a first exemplary case, the second IDD maintains a list of trusted 
DTDs, and the predetermined criteria is equality between the name of the respective DTD and the name of a trusted DTD. Alter- 

2 natively, the predetermined criteria is satisfied by inclusion of the name of a program residing on the second IDD in the respective 
DTD. In a second exemplary case, the transmitting step includes transmitting the generated XML document from the first IDD to the 
second IDD and a third IDD, the operating step includes operating on the data contained in the XML document at the second IDD 
when the respective DTD for the generated XML document satisfies a first predetermined criteria, and the method further includes 

O tne ste P of operating on the data contained in the XML document at the third IDD when the respective DTD for the generated XML 
document satisfies a second predetermined criteria. A system composed of IDDs interconnected by an in home digital network 

^ (IHDN) or other mechanism, which system can be operated by the method mentioned above, is also described. 


WO 01/40938 PCT/EP00/1 141 2 

1 

Methods for initiating activity in intelligent devices connected to an in home digital network 
using extensible markup language (XML) for information exchange and systems therefor 


The present invention relates generally to data communications between 
intelligent information and entertainment digital devices (IDDs) forming an in home digital 
network (IHDN). More particularly, the present invention relates to a novel way to use 
extensible Markup Language (XML) as a mechanism to initiate action in one or more of the 
5 IDDs connected to an IHDN to facilitate interoperability and integration thereof. 

A myriad of different intelligent digital devices (IDDs) for providing 
information and entertainment to the user have come into widespread use. These devices 
include, but are not limited to, conventional (analog) televisions (TVs) and digital TVs 
(DTVs), such as high definition televisions (HDTVs), Direct Broadcast Satellite (DBS) 

10 receivers, cable/terrestrial/satellite set-top boxes (STBs), conventional VCRs, digital video 
recorders (DVRs), telephones, stereo systems, personal computers (PCs), cellular phones, 
DVD players, CD-I players, video game consoles, e.g., the recently introduced Sega 
Dreamcast system, Personal Digital Assistants (PDAs), intelligent Remote Control Devices 
(RCDs), video cameras, digital cameras, etc. All of the above-mentioned IDDs incorporate 

15 one of a Digital Signal Processor (DSP), microcontroller, microprocessor, system-on-a-chip 
(SOC), Programmable Logic Device (PLD), Programmable Logic Array (PLA), or other 
digital data processing means. 

The rank and file of the computer electronics industry in general and the 
software industry in particular have been searching for the next "killer" applications, i.e., an 

20 application that could drive the sale of hardware just like the Visicalc program drove the sale 
of the Apple computer. This killer application, as envisioned by many manufacturers, is a 
single application with a set of features that a consumer would buy for a premium price. A 
single new device may not be this killer application. What may be the application that builds 
customer loyalty and enables widening of profit margins is the ability of a suite of devices to 

25 work together to address the needs of the household as no single device can. This inter- 
operability of devices can increase the value and functionality of the collection of devices in 
the home and can allow these devices, in concert, to provide services that no single device 
can adequately provide. Thus, one possible advance akin to a "killer" application is a 
consumer friendly software set that could be shared by, or distributed among, multiple 
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devices, the ultimate goal being to enable the different intelligent devices to work together in 
a cooperative and integrated fashion, i.e., as if the entire collection of such devices were a 
single, distributed, multi-function device. The holy grail of the "home electronics" industry is 
a mechanism that would allow diverse IDDs to exchange data and act on the exchanged data 
5 in a seamless and transparent manner. Thus, two elements needed to implement this "killer" 
application are a network to connect the IDDs and software permitting the IDDs to act on 
shared information. 

With respect to the first requirement, one way to enable this interconnectivity 
of consumer devices is to have a communications backbone present in the home. It will be 

1 0 appreciated that this backbone, or network, would allow any capable consumer device to 
communicate with any other device attached to the network. Many consumers are already 
beginning to install PC networks in their homes to share, for example, a cable modem or 
digital subscribed line (DSL) fast internet connection. Moreover, in home digital networks 
(IHDNs) have been proposed to interconnect a multiplicity of IDDs. The devices connected 

15 to such an IHDN can communicate with another over any suitable communications medium, 
e.g., via an IEEE 1394 serial communications bus, also known as "Fire Wire", or via a 
wireless communications medium. The IHDN would solve many existing problems of 
combining devices. Connecting a television, a VCR, a satellite dish, a PC, a stereo and six 
speakers would no longer require dozens of cables, multiple splitters purchased at an 

20 electronics store, and a degree in electrical engineering. All that would be required is 

connecting each of these devices to the IHDN and turning it on. However, these connected 
IDDs could still not operate in concert. 

What is needed is a method and corresponding system permitting the exchange 
of information between IDDs. Preferably, what is needed is a method and corresponding 

25 system permitting the exchange of information to initiate an operation responsive to receipt 
of the exchanged information by one of the devices on a network. It would be beneficial if 
the method and corresponding system could operate on a plurality of the exchanged 
information. What is also needed is a method to facilitate data exchanges between 
incompatible intelligent digital devices connected to one another via an IHDN. 

30 


Based on the above and foregoing, it can be appreciated that there presently 
exists a need in the art for a method and corresponding system which increases the 
interoperability of intelligent digital devices connect via an in home digital network. The 
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present invention was motivated by a desire to overcome the drawbacks and shortcomings of 
the presently available technology, and thereby fulfill this need in the art. 

In one aspect, the present invention provides a method of operating a system 
including a digital network interconnected intelligent digital devices (IDDs) generating and 
5 receiving extensible Markup Language (XML) documents containing data and respective 
Document Type Definitions (DTDs) describing the data content. Preferably, the method 
includes steps for transmitting a generated XML document from a first IDD to a second IDD, 
and, when the respective DTD for the generated XML document satisfies a predetermined 
criteria, operating on the data contained in the XML document at the second IDD. In a first 

10 exemplary case, the second IDD maintains a list of trusted DTDs, and the predetermined 

criteria is equality between the name of the respective DTD and the name of a trusted DTD. 
Alternatively, the predetermined criteria comprises the inclusion of the name of a program 
residing on the second IDD. In a second exemplary case, the transmitting step includes 
transmitting the generated XML document from the first IDD to the second IDD and a third 

1 5 IDD, the operating step includes operating on the data contained in the XML document at the 
second IDD when the respective DTD for the generated XML document satisfies a first 
predetermined criteria, and the method further includes the step of operating on the data 
contained in the XML document at the third IDD when the respective DTD for the generated 
XML document satisfies a second predetermined criteria. 

20 In another aspect, the present invention provides a method of operating a 

system including a digital network interconnected intelligent digital devices (IDDs) 
generating and receiving extensible Markup Language (XML) documents containing data 
and respective Document Type Definitions (DTDs) describing the data content. 
Advantageously, the method includes steps for: 

25 (a) generating an XML document containing related data and a reference to a respective DTD 
at a first IDD responsive to a command from a second IDD; 

(b) transmitting the XML document from the first to the second IDD; 

(c) when the respective DTD satisfies a predetermined criteria, parsing the data in the XML 
document in accordance with the format described in the respective DTD to thereby generate 

30 parsed data from the related data; and 

(d) operating on the parsed data. 

In a still further aspect, the present invention provides a system including a 
plurality of intelligent digital devices (IDDs) interconnected to one another , each of the 
IDDs being capable of one of generating and receiving an extensible Markup Language 
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(XML) document containing data and referencing a document type definition (DTD). The 
system is further characterized in that: 

a first IDD generates the XML document responsive to a command received 
over the IHDN; 

5 a second IDD stores N XML processors associated with N named DTDs; 

a third IDD stores M XML processors associated with M named DTDs; 
the second IDD processes the XML document using one of the N XML 
processors when the respective DTD corresponds to one of the N named DTDs; 

the third IDD processes the XML document using one of the M XML 
1 0 processors when the respective DTD corresponds to one of the M named DTDs; and 
N and M are both positive integers. 

Preferably, in the above-mentioned system, at least one of the N named DTDs 
and at least one of the M named DTDs are identical to the respective DTD, while the one of 
the N XML processors corresponding to the respective DTD is different than the one of the 

1 5 M XML processors corresponding to the respective DTD. In an exemplary embodiment, the 
second IDD stores the N named DTDs, while the third IDD stores the M named DTDs. 
Alternatively, the second and third IDDs store lists of trusted DTDs including the associated , 
N and M named DTDs, and the first IDD generates the XML document arid the respective 
DTD responsive to the command received over the IHDN. The IDD's advantageously can be 

20 interconnected via an in home digital network (IHDN). 


These and various other features and aspects of the present invention will be 
readily understood with reference to the following detailed description taken in conjunction with 
25 the accompanying drawings, in which like or similar numbers are used throughout, and in 
which: 

FIG. 1 is a high level block diagram of an intelligent digital device (IDD) 
connected to an in home digital network (IHDN); 

FIGS. 2 A and 2B are alternative configurations of the LUT 128 illustrated in 

30 FIG. 1; 

FIG. 3 is a representation of the entertainment and information system of the 
present invention, particularly illustrating how the invention can be used to exchange 
information based on electronic program guide (EPG) data; 
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FIG. 4 illustrates the format of an XML(EPG) document which can be 
transmitted over the IHDN illustrated in FIGS. 1 and 3; 

FIG. 5 illustrates the format of a document type definition (DTD) associated 
with the XML(EPG) document illustrated in FIG. 4; 
5 FIG. 6 is the output VRML 3D output produced for a sample data set of the 

EPG data in a particular embodiment of the invention; and 

FIG. 7 is a high level flowchart illustrating an exemplary embodiment of the 
method of operating one of the IDDs upon receipt of the XML(EPG) document; 

FIG. 8 shows VRML commands as generated by the parsing of the XML 
1 0 (EPG) document; and 

FIG. 9 is a representational diagram of the multilevel process by which EPG 
data is converted to a 3D EPG display. 


1 5 Illustrative embodiments and exemplary applications will now be described with 

reference to the accompanying drawings to disclose the advantageous teachings of the present 
invention. 

While the present invention is described herein with reference to illustrative 
embodiments for particular applications, it should be understood that the invention is not limited 

20 thereto. Those having ordinary skill in the art and access to the teachings provided herein will 
recognize additional modifications, applications, and embodiments within the scope thereof and 
additional fields in which the present invention would be of significant utility. 

As mentioned above, many manufacturers and device vendors have been 
searching for the next "killer" application for the modern home. One candidate is an in home 

25 digital network (IHDN), which advantageously could be used to interconnect all of the 

accoutrements of a modem home, e.g., DTV, DVD player, surround sound system, several 
PCs, etc. This integrated entertainment and information system is envisioned to include 
components which will exchange data, and preferably act on the exchanged data, in a manner 
which is completely invisible to the system user(s). However, when such a system is 

30 fabricated from current electronic components, the system still has the limitation that not 

every device connected to the IHDN will understand how to communicate with other devices 
attached to the IHDN. 

In terms of the integrated entertainment and information system, once all of 
the devices in the home are connected together, either hardwired or by a broadcast 
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connection, a communications standard, or common language, is still required so that all of 
the devices can be understood by all the other devices, i.e., so the connected devices can 
exchange information and cooperate. The language must be: 
Flexible enough to support a wide array of devices; 
5 Extensible, so new classes of devices can be supported; 

Easily Parsed by processor and memory constrained devices; and 
An open standard with broad industry backing. 

The present invention solves this problem by employing a method in which 
XML, a language which possesses all of the qualities required in a communications standard 

10 for the IHDN, is used in a new way to solve the problem of device interconnectivity. In other 
words, the method and corresponding system according to the present invention overcome 
the above-identified problem by fabricating the system from intelligent digital devices 
(IDDs). In the inventive system, each of the IDDs is capable of generating and/or receiving 
an extensible Markup Language (XML) document. Moreover, IDDs capable of receiving 

1 5 the XML document store at least one XML application for processing the data contained in 
the XML document in accordance with a document type definition (DTD). These features of 
each IDD will be discussed in greater detail below. Before a detailed discussion of the IDD is 
undertaken, a brief discussion of the XML standard, which is a subset of the SGML standard, 
will be presented. 

20 SGML, an acronym for Standard Generalized Markup Language, is an 

international standard (ISO 8879:1986) and is a markup language used extensively in 
publishing and other domains. Additional material on SGML is included in the document 
entitled "A Gentle Introduction to SGML", Text Encoding Initiative, which can be obtained 
from the web page http://www-tei.uic.edu/orgs/tei/sgml/teip3sg/index.html. Moreover, XML 

25 is a markup language based on SGML. In either case, "markup" denotes that the information 
is annotated (or marked up) with tags. Tags surround a section of information (called content) 
and define the properties of the content. A more detailed explanation is provided in the 
article by M. Bryan, entitled "An Introduction to the Extensible Markup Language (XML)", 
which is available from The SGML Centre, at http://www.personal.u- 

30 net.com/~sgml/xmlintro.htm; and the paper by Jon Bosak entitled "XML, Java, and the 
Future of the Web", which is available from Sun Microsystems at 
http://metalab.unc.edu/pub/sun-info/standards/xml/why/xmlapps.htm. 

The XML standard is the result of an effort by the World Wide Web 
Consortium (W3C) to enable the further expansion of Web technology into new domains of 
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distributed document processing. The first officially released specification for XML from 
W3C was released on Feb. 10, 1998 and the specifications for required related standards are 
still being formulated. 

XML is used to capture structured data and text. In contrast, Hypertext 
5 Markup Language (HTML), the language used to develop Web pages, is capable of capturing 
multimedia documents. Any inherent relationship between the information captured in an 
HTML document is lost. For example, text annotated in HTML could have the form <bold> 
text </bold>, indicating that the included text would be displayed in a bold font. In contrast, 
data annotated in XML would have the form <program-type> text </program-type>, 

1 0 indicating that the data between the tags denotes the genre of a movie in an electronic 
program guide (EPG)). It is important to note that in XML, these data relationships are 
preserved and can be used by the receiver of the XML document. It should also be mentioned 
that there are several related standards for XML that specify how to display and link XML 
documents; this suite of XML standards, used together, can capture, display and link together 

1 5 highly structured information in an effective way. 

As mentioned immediately above, an important feature of XML is that it 
separates the data from how that data should be displayed. This is useful because it allows the 
same XML document to be used by multiple IDDs for different purposes. Since each IDD 
advantageously can display the data in an appropriate manner, the XML document is flexible 

20 and reusable. 

It should be mentioned that the manner is which each IDD employs the XML 
documents it receives is determined by the Document Type Definition (DTD). The DTD 
defines the structure of the XML document including what each tag represents. It should also 
be mentioned that each IDD advantageously can have its own set of DTD(s) that is (are) 

25 applied to the XML documents that the IDD receives. Moreover, as mentioned below, the 
referenced DTD in an XML document advantageously can be employed to initiate operation 
of a specific program residing on a respective IDD on the received XML document. See Fig. 
7. It will be appreciated that several IDDs can have different versions of, for example, a 
VRML interpreter, each version generating a VRML world, as discussed in greater detail 

30 below with respect to Fig. 6, sized to a different display device. In this way, each IDD can 
use the same XML document and associated DTD to produce, for example, an IDD-specific 
presentation. Moreover, as discussed in greater detail below, substantially identical versions 
of a XML document advantageously can be formatted for display and presentation, in an 
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exemplary case, by merely editing the DTD name included in the XML document to thereby 
instantiate a different XML application. 

The ability of the IDDs to generate, transmit, receive, and process XML 
documents in accordance with respective DTDs are features essential to creating a flexible 
and scalable IHDN. The entertainment and information system according to the present 
invention thus consists of a plurality of IDDs connected through a network and 
communicating with one another using XML documents. As discussed in greater detail 
below, each IDD contains an XML parser or processor that allows the IDD to read an XML 
document and a DTD (or set of DTDs) that allows the IDD to understand XML documents 
received across the network. 

A high level block diagram of an exemplary IDD 100 is illustrated in FIG. 1 as 
including processor 110 receiving instructions from a memory 120 via a bus 130. It will be 
appreciated that the memory 120 advantageously can include a read-only memory (ROM) 
122, a random access memory (ROM) 124, and a general purpose memory 126, which could 
be a hard disk drive, a CD-ROM, etc. It will also be appreciated that the bus 130 can include 
a number of general and special purpose buses known to one of ordinary skill in the art. The 
processor 1 1 0 preferably is connected to the IHDN 200 via a first input/output (I/O) device 
140 and is connected to another device, e.g., a display 160, via a second I/O device 150. 

Preferably, the memory 120 stores at least one XML application for generating 
and parsing XML documents. Advantageously, the XML application may also build or 
facilitate the building of XML documents and associated DTDs. In addition, the memory 
stores the operating program(s) of the IDD, which program(s) advantageously can include(s) 
a routine for instantiating the XML application when (a) an XML document has been 
received by the IDD 100 and (b) a predetermined condition is satisfied. 

In an exemplary case, the 120 stores a look up table (LUT) 128 employing the 
names of trusted DTDs at pointers to respective XML applications, where trusted DTDs are 
DTDs that have tested satisfactory when communicating between IDD 100a and 100b, for 
example. It will be appreciated that trusted DTDs need not be supplied by the vendor of the 
specific IDD. It will also be appreciated that the memory 120 need not store the actual 
trusted DTD; the associated XML program can be instantiated as long as a match occurs 
between the DTD referenced in the XML document and the name of the trusted DTD stored 
in LUT 128. Alternatively, the memory 120 advantageously can store the trusted DTDs, 
which saves bandwidth in the IHDN 200. See FIGS. 2A and 2B. 
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As previously discussed, the present invention employs XML documents as a 
data exchange mechanism, allowing various and diverse IDDs 100 to share data between 
themselves over the IHDN 200. One exemplary embodiment of an entertainment and 
information system 1 according to the present invention is illustrated in FIGS. 3-9 with respect 
5 to conveying Electronic Program Guide (EPG) data between several IDDs over the IHDN 200. 

Referring to FIG. 3, it will be appreciated that one way to conceptualize the 
entertainment and information system 1 is by means of the display-centric model being 
investigated by one of the High Volume Electronics (HVE) strategy groups working in 
consumer electronics product planning. In this model, a consumer buys a display IDD 100a, 

10 e.g., a flat panel display, an entertainment IDD 100b, e.g.. a cable/terrestrial/satellite receiver 
(a.k.a. set- top-box (STB)) which performs all of the audio/video/data decoding functions, an 
information IDD 100c, e.g., WebTV or similar technology for obtaining Internet content, a 
recorder IDD lOOd, e.g., a digital video recorder or a RePlay recorder, and a computer IDD 
lOOe, i.e., a personal computer. All of the IDDs lOOa-lOOe are operatively connected to one 

15 another via the IHDN 200. It will be appreciated that the information IDD 100c can be 

connected to the Internet (400) via a plain old telephone system (POTS) 300, so that data can 
be gathered from various data sources, e.g., 402A-402C. 

It should be noted that several of the IDDs, e.g., entertainment IDD 100b and 
information IDD 100c advantageously can be combined in a single IDD lOOx, although these 

20 IDDs are shown separately in FIG. 3. It should also be noted that the notation IDD is 
employed throughout to denote the wide scope of the present invention. For example, a 
nourishment IDD lOOf. e.g., a microwave oven, and a speaker system IDD lOOg 
advantageously can be connected to and form part of the entertainment and information 
system 1 according to the present invention without departing from the spirit and scope of the 

25 invention. Moreover, it will be appreciated that the interconnection of multiple IDD's 100 via 
the IHDN 200 represents only a single preferred embodiment according to the present 
invention. The IDD ! s 100 advantageously can be interconnected via the internet, a wireless 
network, a local area network, a wide area network or a virtual private network. 

In the exemplary preferred embodiment of the invention illustrated in FIG. 3, 

30 the entertainment IDD 100b, e.g., the STB, receives raw Electronic Program Guide (EPG) 

data sent by the broadcaster or cable operator as part of the Program and System Information 
Protocol (PSIP) standard. See FIG. 9. The EPG data is saved in a database stored in memory 
120b located in the IDD 100b. The processor 1 10b in the IDD 1 00b then constructs an XML 
document model out of the raw data, i.e., the EPG data retrieved from the database in 
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memory 120b, and transmits the XML packaged EPG data (hereinafter XML(EPG) 
document) to the display IDD 1 00a. In the exemplary embodiment being discussed, the 
XML(EPG) document includes a reference to a Virtual Reality Modeling Language (VRML) 
DTD, as illustrated in FIG. 4; the exemplary VRML. DTD is illustrated in FIG. 5. As 
5 previously mentioned, the IDD 100b need not actually generate the VRML. DTD (although 
the IDD 100b could generate the applicable DTD when required). The XML(EPG) document 
(and the VRML. DTD) is (are) transmitted to the IDD 100a, where the XML(EPG) document 
is converted for display on display 160a via the VRML 3D encoder instantiated by processor 
1 10a. A representative screen capture of the VRML output is illustrated in FIG. 6. 

1 0 Referring now to FIG. 7, the method for operating the entertainment and 

information system 1 will now be explained in detail. More specifically, FIG. 7 is a high 
level flowchart of a method for operating the entertainment and information system 1 
according to the first preferred embodiment of the present invention. During step S10, 
XML(EPG) document is received at IDD 100a from IDD 100b via IHDN 200. Then, during 

15 step S20, a check is performed to determine whether the XML(EPG) document references a 
trusted DTD, i.e., the VRML. DTD, which advantageously can be listed in LUT 128 of IDD 
100a. It will be appreciated from FIGS. 2A and 2B that it is immaterial to the present , 
discussion whether the trusted DTD referenced in the XML(EPG) document is transmitted 
via the IHDN 200 or stored in memory 120 of IDD 100a. When the answer is negative, the 

20 operating program of the IDD 100a jumps to step S50 and performs a check to determine: 
whether the XML(EPG) document references an EPG.DTD. 

If the answer at step S20 is affirmative, the processor 110 instantiates the 
XML processor stored in memory 120 during step S30 and instantiates the VRML interpreter 
stored in memory 120 during step S40. The XML processor parses the EPG data in the 

25 XML(EPG) document and outputs selected portions of the EPG data to the VRML 

interpreter, which, in turn, interprets the parsed EPG data and generates VRML data, as 
illustrated in FIG. 8, representing a three dimensional (3D) VRML "world" projected on the 
screen of the IDD 100a. See FIG. 6, which, as previously mentioned, is a screen capture of a 
VRML world depicting selected portions of the EPG data, as discussed in greater detail 

30 below. 

When the answer is affirmative at step S50, the processor 1 10 instantiates an 
XML application during step S60, which parses the EPG data in the XML(EPG) document 
and outputs selected portions of the EPG data to the screen of the IDD 100a as EPG data in a 
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grid form. When the answer is negative at step S50, the routine loops back to the start (step 
S10) to await the arrival of the next XML document. 

Similarly, the IDD 100c, e.g., the WebTV unit or similar component, 
advantageously can receive Web content in XML or convert HTML into XML/DOM 
5 (Document Object Model). The IDD 100c can transmit the XML(HTML) document to the 
IDD 100a for display. Likewise, the recorder IDD lOOd, e.g., a video recorder, also 
connected to other devices via the IHDN 200, transmits its table of contents (TOC) in XML 
format, i.e., transmits an XML(TOC) document to the IDD 100a for conversion to VRML for 
display. Thus, the IDD 100a, which advantageously includes an XML parser and a VRML 
10 decoder (interpreter and 3D pipeline), can interact with any other IDD to display data 
received in XML format. 

It will be appreciated that the present invention is not limited to the exemplary 
embodiment described above. For example, it will be appreciated that IDD 100a 
advantageously can display 3D data representing the output of a plurality of data sources, 

1 5 e.g., IDDs lOOb-lOOd, so that the user(s) can view and information residing on or obtainable 
via the entertainment and information system 1 . Moreover, it will be appreciated that since 
the reference to a trusted DTD can be the trigger which initiates the operation of an XML 
processor or XML application, receipt of two such XML documents or DTDs advantageously 
can signal the initiation of a third XML application. In an exemplary case, when the 

20 XML(EPG) and XML(TOC) documents are both received by the IDD 1 00a, an exemplary 
XML application called SORT can be instantiated to examine the XML(EPG) document 
corresponding to the displayed VRML world for coincidences and/or discontinuities. In an 
illustrative case, the SORT XML application could be employed to determine that a 
particular program was marked as being of interest to the user but has not been designated for 

25 recording by recorder IDD lOOd. It will be appreciated that an updated TOC for the IDD 
lOOd advantageously could be transmitted from the IDD 100a to the IDD lOOd as a second 
XML(TOC) document. 

In order to better understand the IHDN described in this particular 
embodiment of the invention, a more detailed discussion regarding the movement of 

30 Electronic Program Guide (EPG) data from IDD to IDD will be presented. It will be 
appreciated that EPG applications will be more prevalent both as DTV becomes more 
prevalent and as the number of channels available to the user continues to grow. It will be 
appreciated that current two dimensional (2D) method of formatting data for display to the 
user do not scale up to the eventual 500+ channels predicted for DTV. 


WO 01/40938 PCT/EP00/11412 

12 

The EPG-VRML application discussed above is a three dimensional (3D) 
representation of the programs available in a graphic format similar to the layout appearing in 
most U.S. newspapers. The advantage of this representation over the 2D representation is that 
the 3D representation is scalable, i.e., the 3D display advantageously can accommodate the 
5 large numbers of channels that will be available to consumer in the near future. As mentioned 
above, the application was developed using VRML to display the EPG. 

It should be mentioned that the EPG data is a structured data set which is very 
large. Advantageously, XML is designed to work with structured data and can handle 
arbitrarily large documents. Moreover, the EPG data is composed of text based information; 

10 XML works with text based documents. It will be appreciated that XML documents can 
reference files containing non-textual data, i.e., images, sounds, etc. 

The XML EPG document must capture all of the information necessary to 
display both a concise listing of the programming available and a more verbose descriptive 
listing of the programs available. A Document Type Definition (DTD), as documented in . 

1 5 XML literature, describes the data to be included in the XML(EPG) document. This DTD is 
depicted in FIG. 5. It will be noted that the VRML.DTD indicates the name of the processing 
application which advantageously can be used to further manipulate the EPG data, as 
discussed in greater detail below. 

It will be noted that the DTD describes how all of the individual data elements 

20 associated with each television program are composed and related to each other. The DTD 
also describes how all of the television programs are grouped together to create an 
XML(EPG) document that captures all of the available programming on all of the available 
channels for some predetermined time period. Preferably, the XML(EPG) document must 
contain specific information about each program available to be useful to a user. The EPG 

25 document must, at a minimum contain the name of each available television program, the 
start time, the end time, and the channel for every television program. Additionally, each 
television program in the XML(EPG) document has a broad category label, indicating the 
type of program, i.e., comedy, musical, etc. A description of each television program would 
also be useful to the user, and as such is included in the XML(EPG) document and the 

30 associated DTD. As a possible extension, additional category specific information can also be 
included with the television program. For example, a movie listed in the EPG data 
advantageously can indicate the name of the director while a sporting event would indicate 
the type of sport being played and names of the players or teams playing. 
As mentioned above, one partial example of the XML(EPG) document is depicted in FIG. 4. 
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It should be noted from FIG.4 that the name of the channel associated which 
this television program may not be needed by the EPG. In that case, the channel 
identification will simply be discarded when the IDD 100a parses the XML(EPG)document 
and displays the 3D representation of the EPG data to the user. 
5 It will be appreciated that the data representation in XML is highly scalable. 

Any number of programs on any number of channels can be added to the XML document and 
transmitted to the EPG application via entertainment IDD 100b, IHDN 200, and display IDD 
100a, which would cooperatively take the EPG data, parse it, and display the 3D 
representation of the EPG data, i.e., the VRML world, for the user. XML can therefore be 

10 used as the data transmission facilitator, even as the number of channels and programs 
increases over time. 

Once the XML(EPG) document is received by the XML processor and VRML 
interpreter instantiated by processor 1 10a, the document is processed and displayed. For this 
application, the EPG data within the XML(EPG) document is converted to VRML data by 

15 the VRML interpreter and displayed for the user to browse and navigate. The parsing of the 
EPG data in the XML(EPG) document and the conversion of the parser's output to VRML 
format is performed by the XML processor and VRML encoder running on processor 1 10a. It 
is important to note that, because the EPG data is encoded in XML, a different XML 
application corresponding to a different method of displaying the EPG data could be 

20 substituted for this application (See FIG. 7.) without requiring any changes to the XML(EPG) 
document or the transmission mechanism, with the possible exception of modifying the DTD 
name in the XML(EPG) document. In short, an XML application is simply a consumer of the 
XML(EPG) document and does not need to know anything about the origins of the 
XML(EPG) document or how this document was delivered to the XML application. The 

25 originator of the EPG data in the XML(EPG) document could also be replaced without 
affecting the display of EPG data in any meaningful way. 

It will be appreciated that parsing the XML(EPG) document entails walking 
through the XML document and collecting up all the relevant information, in an exemplary 
case, about each of the programs in the XML document. This information must then be 

30 translated into a useful VRML representation of the data, which is a relatively 

straightforward transform understood by one of ordinary skill in the art. A selected portion of 
the VRML program data is shown in FIG. 6; a selected portion of the VRML commands is 
depicted in FIG. 8. 
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The EPG VRML program data is used by the EPG application to display all of 
the programs in the VRML visual display to the user. This allows the data to change, but 
keep the same visual VRML representation. 

A functional diagram depicting the entire process, from receiving EPG data at 
5 IDD 100b to displaying the 3D visualization of the EPG data on display IDD 100a 

application, including an exemplary data transformation, transmission, and display sequence 
is illustrated in FIG. 9. In FIG. 9, it is assumed the raw EPG data is not saved in XML 
format and must be converted to XML by IDD 1 00b. Before the data is converted to XML, it 
is cleansed of any erroneous or unnecessary data points and saved in memory 120b of IDD 
10 100b. The cleansed data can then be formatted into a structure that makes conversion into 

XML easier. It will be appreciated that this may involve ordering the data in a consistent way 
and collecting related data records together. Once this step has been completed, the data is 
converted to and XML representation, based on the selected DTD for EPG data, by processor 
110b. 

15 The XML(EPG) document is then transmitted to the display IDD 100a for 

display. Once received by the IDD 100a, the XML(EPG) document is parsed and converted 
into the format necessary for display, using the referenced DTD. See FIG. 7. In the 
exemplary case under discussion, the XML processor parses the XML(EPG) document and 
generates data suitable for application to the VRML interpreter, which generates the VRML 

20 data corresponding to the EPG data. This VRML code is then combined with the VRML 
code that describes the 3D VRML world structure. Once this step is completed, a VRML 
world containing a 3D representation of current EPG data is ready for display on IDD 100a. 

It will be appreciated from FIG. 7 that the VRML world has many features to 
aid the user in navigating the EPG data in a useful manner. The user can visually highlight 

25 categories of programs, programs within specific time periods, or some combination of the 
two. The user may also select channels of interest and move up and down through the vast 
selection of available channels. It is also possible to display a preview of some of the 
programs on the preview screen above the EPG program listings. It will be noted that the day 
tower at the nexus of the channel axis and time axis is used to change the day being viewed. 

30 It should be noted again that all of the documents, articles and papers cited 

above are incorporated herein by reference. Moreover, is should again be stressed that while 
the mechanism by which the various IDD's are interconnected is described as an IHDN with 
respect to one preferred embodiment, the present invention is not limited to IDD's 
interconnected in this manner. The IDD ! s can also be interconnected via the internet, a 
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virtual private network, a wireless network, a local area network, or a private wide area 
network without departing from the scope of the present invention. 

Finally, it should be noted that many of the tools currently available for 
working with XML can be obtained for free. Many are simple experiments by individuals 
5 interested in exploring this new domain, some are beta versions of tools that companies plan 
on selling at some point in the future, and other tools are released by companies as freeware 
in the hopes that it will spur the movement of development efforts on the Internet towards 
XML and other products sold by these companies. 

Although a presently preferred embodiment of the present invention has been 
1 0 described in detail hereinabove, it should be clearly understood that many variations and/or 
modifications of the basic inventive concepts herein taught, which may appear to those 
skilled in the pertinent art, will still fall within the spirit and scope of the present invention, as 
defined in the appended claims. 
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1 . A method of operating a system including a digital network interconnected intelligent 
digital devices (IDDs) generating and receiving extensible Markup Language (XML) 
documents containing data and respective Document Type Definitions (DTDs) describing the 
data content, comprising: 

5 transmitting a generated XML document from a first IDD to a second IDD; and 

when the respective DTD for the generated XML document satisfies a predetermined 
criteria, operating on the data contained in the XML document at the second IDD. 

2. The method as recited in claim 1, wherein the second IDD maintains a list of trusted 
1 0 DTDs and wherein the predetermined criteria is equality between the name of the respective 

DTD and the name of a trusted DTD. 

3. The method as recited in claim 1, wherein: 

the transmitting step comprises transmitting the generated XML document from the 
1 5 first IDD to the second IDD and a third IDD; 

the operating step comprises operating on the data contained in the XML document at 
the second IDD when the respective DTD for the generated XML document satisfies a first 
predetermined criteria, and 

the method further comprises the step of operating on the data contained in the XML 
20 document at the third IDD when the respective DTD for the generated XML document 
satisfies a second predetermined criteria. 

4. The method as recited in claim 3, wherein: 

the second IDD maintains a first list of trusted DTDs; 
25 the third IDD maintains a second list of trusted DTDs; 

the first predetermined criteria is equality between the name of the respective DTD 
and the name of a trusted DTD on the first list; and 

the second predetermined criteria is equality between the name of the respective DTD 
and the name of a trusted DTD on the second list. 
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5. The method as recited in claim 3, wherein the XML document and the respective 
DTD are transmitted to the second and third IDDs. 

5 6. The method as recited in claim 3, wherein the respective DTD is stored on at least one 
of the second and third IDDs. 

7. A method of operating a system including a digital network interconnected intelligent 
digital devices (IDDs) generating and receiving extensible Markup Language (XML) 
1 0 documents containing data and respective Document Type Definitions (DTDs) describing the 
data content, comprising: 

(a) generating an XML document containing related data and a reference to a respective 
DTD at a first IDD responsive to a command from a second EDD; 

(b) transmitting the XML document from the first to the second IDD; 

15 (c) when the respective DTD satisfies a predetermined criteria, parsing the data in the 

XML document in accordance with the format described in the respective DTD to thereby 
generate parsed data from the related data; and 
(d) operating on the parsed data. 

20 8. The method as recited in claim 7, wherein: 

the second IDD stores a list of trusted DTDs associated with respective XML 
processors; 

the predetermined criteria is coincidence between the respective DTD and a trusted DTD 
on the list; and 

25 the parsing and the operating steps are performed using the one of the XML processors 

corresponding to the respective DTD. 

9. The method as recited in claim 8, wherein: 

the second EDD stores a plurality of DTDs and associated XML processors; 
30 the XML document references the respective DTD; and 

the parsing and the operating steps are performed using the one of the XML processors 
corresponding to the respective DTD. 


10. A system comprising: 


WO 01/40938 PCT7EPO0/11412 

18 

a plurality of intelligent digital devices (IDDs) (100) interconnected to one another, 
each of the IDDs being capable of one of generating and receiving an extensible Markup 
Language (XML) document containing data and referencing a document type definition 
(DTD); wherein: 

a first IDD generates the XML document responsive to a command received over the 

IHDN; 

a second IDD stores N XML processors associated with N named DTDs; 

a third IDD stores M XML processors associated with M named DTDs; 

the second IDD processes the XML document using one of the N XML processors 
when the respective DTD corresponds to one of the N named DTDs; 

the third IDD processes the XML document using one of the M XML processors 
when the respective DTD corresponds to one of the M named DTDs; and 

N and M are both positive integers. 

1 1 . The system as recited in claim 1 0, wherein at least one of the N named DTDs and at 
least one of the M named DTDs are identical to the respective DTD, and wherein the one of 
the N XML processors corresponding to the respective DTD is different than the one of the 
M XML processors corresponding to the respective DTD. 

12. The system as recited in claim 10, wherein the second and third IDDs store lists of 
trusted DTDs including the associated N and M named DTDs, and wherein the first IDD 
generates the XML document and the respective DTD responsive to the command received 
over the IHDN. 
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<?xml versions. 0"?> 

<!DOCTYPE guide SYSTEM "VRML.DTD"> 


<EPG> 


<Date>06-04-1998</Date> 
<TimeBlock Block ='6pm-midnighl"> 
<channel> 

<definitionxnurnber>2</numberxname>CBS</name> 


<channel-type>Network<channel-type></definilion> 


<movie> 

<tille>Miller's Crossing</title> 

descriptions noir gangster movie that examines a conflict 

between the Irish and Italian mafia in a small city in 1920s 

America. This interesting, but very violent tale follows a 

top member of the Irish gang as he switches sides and 

explores a web of deceit and double-crosses between members 

of the lwo gangs, the corrupt local government and other 

shady characters.</description> 

<program-type>Drama</pgram-type> 

<program-sublype>Gangster</program-subtype> 

<running-time>2:00</running-time> 

<review-rating>*"</review-rating> 

<language>EnglislWlanguage> 

<release-year>1991</release-year> 

<players> 

<directorxloel Coen</director> 
<producer>Elhan Coen<yproducer> 
<actor>John Tuturro</actorx/players> 

</movie> 

<parental-rating>TVMA</parental-rating> 



</Entry> 
</channel> 
</TimeBlocb 


</Program> 


</EPG> 
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<!- DTD for a Simple TV Guide Application R. Krasinski 1998.06.04 --> 

< ! ELEMENT TVGuide - - (Date, TimeBlock+) > 

< ! ELEMENT Date - - (/PCDATA) > 

< ! ELEMENT TimeBlock - - (channel)+> 

< ! ATTLIST Block CDATA /REQUIRED > 

< ! ELEMENT channel - - (definition, Entry+)+> 

< ! ELEMENT definition - - (number, name, channel-type)> 

< ! ELEMENT Entry - - (Time-slot, program) +> 

< ! ELEMENT Time-slot - - (start-time, end-time)> 

< ! ELEMENT program - - ((movie I series-episode I sport I talk I news I cartoon), parental-rating))> 
< ! ELEMENT movie - - (title, description, program-type, program-subtype, running-time, review-rating, 
language, release-year, MPAA-rating, players) > 

< ! ELEMENT players - - (director, producer, actor*) > 

< ! ELEMENT actor - - (guild-name, award) > 
< ! ELEMENT producer - - (person-name, award) > 
< ! ELEMENT director - - (guild-name, award) > 
< ! ELEMENT parental-rating - - (/PCDATA) > 
<! ELEMENT number-- (/PCDATA) > 
<! ELEMENT name- (/PCDATA) > 
< ! ELEMENT channel-type - - (/PCDATA) > 
<! ELEMENT start-time- (/PCDATA) > 
<! ELEMENT end-time-- (/PCDATA) > 
<! ELEMENT title- (/PCDATA) > 
<! ELEMENT description- (/PCDATA) > 
< ! ELEMENT program-type - - (/PCDATA) > 
< ! ELEMENT program-subtype - - (/PCDATA) > 
< ! ELEMENT running-time - - (/PCDATA) > 
< ! ELEMENT review-rating - - (/PCDATA > 
<! ELEMENT language- (/PCDATA) > 
<! ELEMENT release-year -- (/PCDATA) > 
< ! ELEMENT MPAA-rating - - (/PCDATA) > 
<! ELEMENT guild-name-- (/PCDATA) > 
<! ELEMENT person-name - (/PCDATA) > 
<! ELEMENT award- (/PCDATA) > 


FIG. 5 
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<!-- DTD lor a Simple TV Guide Application R. Krasinski 1998.06.04 -> 

< ! ELEMENT TVGuide - - (Dale, TimeBlock+) > 

< ! ELEMENT Date - - (/PCDATA) > 

< ! ELEMENT TimeBlock - - (channel)+> 

< ! ATTUST Block CDATA /REQUIRED > 

< ! ELEMENT channel - - (definition, Entry+)+> 

< ! ELEMENT definition - - (number, name, channel-type) > 

< ! ELEMENT Entry - - (Time-slot, program) +> 

< ! ELEMENT Time-slot - - (start-time, end-lime)> 

< ! ELEMENT program - - ((movie I series-episode I sport I talk I news I cartoon), parental-rating))> 
< ! ELEMENT movie - - (title, description, program-type, program-subtype, running-time, review-rating, 
language, release-year, MPAA-rating, players) > 

< ! ELEMENT players - - (director, producer, actor*) > 

< ! ELEMENT actor - - (guild-name, award) > 
< ! ELEMENT producer - - (person-name, award) > 
< I ELEMENT director - - (guild-name, award) > 
< ! ELEMENT parental-rating - - (/PCDATA) > 
< ! ELEMENT number - - (/PCDATA) > 
<! ELEMENT name-- (/PCDATA) > 
<! ELEMENT channel-type - (/PCDATA) > 
<! ELEMENT start-time - - (/PCDATA) > 
<! ELEMENT end-time-- (/PCDATA) > 
<! ELEMENT title-- (/PCDATA) > 
<! ELEMENT description-- (/PCDATAb 
< ! ELEMENT program-type - - (/PCDATA) > 
< ! ELEMENT program-subtype - - (/PCDATA) > 
< ! ELEMENT running-time - - (/PCDATA) > 
< ! ELEMENT review-rating - - (/PCDATA > 
<! ELEMENT language- (/PCDATA) > 
<! ELEMENT release-year-- (/PCDATA) > 
< ! ELEMENT MPAA-rating - - (/PCDATA) > 
<! ELEMENT guild-name- (/PCDATA) > 
< ! ELEMENT person-name - - (/PCDATA) > 
<! ELEMENT award-- (/PCDATA) > 


FIG. 8 
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